1. Field of the Invention
The present invention relates generally to backup storage systems, and in particular to deduplication based systems storing backup data from multiple clients.
2. Description of the Related Art
Backup storage servers perform multiple tasks, and ideally perform the tasks quickly and efficiently. The tasks may include receiving data from clients, storing data, maintaining a database of metadata (sometimes referred to as a “metabase”), transferring data between separate storage devices, and restoring data to clients. At times, clients may be restricted from sending data for backup if the other tasks are dominating the resources of the backup storage server. Preferably, the periods of time when the backup server is unable to receive new client data that a client desires to send should be minimized.
In a typical deduplication based backup system, many clients may be sending data segments to a deduplication backup server at the same time. The backup server may store segments from each individual client in a relatively localized manner in a backup storage medium to facilitate future restore operations, since segments from the same client have spatial locality and may be read together during restore time. Keeping data for each client together (generally speaking) may become a challenge when data from multiple clients is being received and processed concurrently.
Backup servers also confront other problems when receiving and processing data for multiple clients concurrently. For example, backup servers may be unable to receive new backup data from clients while committing transactions involving already received backup data. The server may commit a transaction from a client after all of the data from that client for a particular backup transaction has been received by the server. Committing the transaction may include flushing and synchronizing all open container files and container index files, closing the container and container index files, and updating the backup server's metadata (e.g., metabase). The metabase may contain information about which segments belong to which client, their location in the storage medium, and links to identical segments in the storage medium from segments that have been deduplicated.
In some embodiments, the receiving server may use one or more buffers to temporarily store received data. During the transaction commit time, the server may be unable to move data out of the temporary buffers in system memory to the backup storage medium. If the commit time is long enough, the buffers may reach maximum capacity and not be able to store newly received data. To prevent the buffers from overflowing, clients may be prevented from sending new data to the backup server. Therefore, if the amount of time spent by the backup server committing transactions can be reduced, the amount of time clients are restricted from sending backup data to the backup server may also be reduced.
In view of the above, improved methods and mechanisms for managing data storage are desired.